- Deep learning
- Feed-forward neural networks
- Recurrent neural networks
A machine learning subfield of learning representations of data. Exceptional effective at learning patterns.
Deep learning algorithms attempt to learn (multiple levels of) representation by using a hierarchy of multiple layers.
\[h = \sigma(W_1x + b_1)\] \[y = \sigma(W_2h + b_2)\]
Learned hypothesis may fit the training data very well, even outliers ( noise) but fail to generalize to new examples (test data)
How to avoid overfitting?
# Use Keras Functional API
input <- layer_input(shape = list(maxlen), name = "input")
model <- input %>%
layer_embedding(input_dim = max_words, output_dim = dim_size, input_length = maxlen,
weights = list(word_embeds), trainable = FALSE) %>%
layer_lstm(units = 80, return_sequences = TRUE)
output <- model %>%
layer_global_max_pooling_1d() %>%
layer_dense(units = 1, activation = "sigmoid")
model <- keras_model(input, output)
summary(model)
# instead of accuracy we can use "AUC" metrics from "tensorflow.keras"
model %>% compile(
optimizer = "adam",
loss = "binary_crossentropy",
metrics = tensorflow::tf$keras$metrics$AUC() # metrics = c('accuracy')
)
history <- model %>% keras::fit( x_train, y_train, epochs = 10, batch_size = 32, validation_split = 0.2 )